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ABSTRACT 

This paper examined state and community programs for 
preschool-aged children in Illinois during a 1-year period in order 
to identify, describe, and publicize effective practices in early 
childhood education. Exemplars, rather than content-specific 
standards, were used for selection criteiia. A qualitative, 
connoisseurship model of evaluation was used by teeuns of experts in 
early childhood education. The nine programs that were selected as 
finalists represented four practice areas: (1) staffing patterns; (2) 
service delivery; (3) family involvement; and (4) program design. It 
is concluded that the findings may move the state closer to an 
understanding of commonalities in programs for young children that 
exist across ability groupings, service providers, and geographic 
areas. The use of content experts in the planning, design, and 
conduct of the evaluation helped build credibility for the 
evaluation. Case study methodology was used successfully, and case 
study information will be used in the resource directory that will be 
made available to persons interested in implementing practices in 
other areas- It was found that local programs either did not have 
access to data concerning the effectiveness of their programs, or 
they lacked the skills, resources, and encouragement needed to 
evaluate their practices. (SH) 



f Reproductions supplied by EDRS are the best that can be made 
* from the original document. 



EDUCATlONAt RESOURCES iNFORMATfON 
CENTER ^ERID 

Np^Thift <locum«fif hit bt«r> r«oroduc«d n 
Ttc»tv«cf from tf»« p«f«or» o* orQamtfttton 

O Minor Ching«» bM^ miOt fO improve 
0£R» po«ffK>ft Of poitcy 



00 
00 



A STATEWIDE SEARCH FOR EXEMPLARY PRACTICES 
IN EARLY CHILDHOOD EDUCATION IN ILLINOIS 



CD 



O 



Lizanne DeStefano, Ph.D. 
Departoieni of Edu';ational Psychology 
University of Illinois at Urbana-Champaign 

Susan P. Maude, Ph.D. 
Department of Specif Education 
University of Illinois at Urbana-Champaign 

Sandra Heinzel Crews 
Illinois State Board of Education 
Springfield, IL 

and 

Linda Mabry, Loctoral Candidate 
Department of Educational Psychology 
University of Illinois at Urbana-Champaign 



Paper presented at the Annual Meeting of the 
American Education Research Association 
Chicago, IL, April 3-7, 1991. 



PERMISSION TO REPRODUCE THIS 
MATERIAL HAS BEcN GRANTED BY 



TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC) " 



ERIC 



I 

BEST COPY AMABi^ 



A STATEWIDE SEARCH FOR EXEMPLARY PRACTICES 
IN EARLY CHILDHOOD EDUCATION IN ILLINOIS. 



Lizanne DeSiefano, Ph.D., Susan P. Maude, Ph.D., Sandra Heinzel Crews, and Linda Mabry 
University of Illinois. Urbana-Champaign and the Illinois State Board of Education 

I. Background of the study 

Early Childhood (EO Legislation 

Supporting policy for the education of very young children with and without disabilities 
has been well documented over the years (Allen, 1984; Hanson & Lynch, 1989; Peterson, 1987; 
Shonkoff & Meisels, 1990). As early as 1935, funding for innovative programs was provided 
through Title V, using Special Projects of Regional and National Significance (SPRANS) grants 
(Shonkoff & Meisels, 1990). With the passage of the Medicaid provisions of the Social Security 
Act of 1965, early medical, health prevention, and intervention programs for poor children were 
established. Also in 1965, policy enacted through the Elementary and Secondary Education Act 
established community-based Head Start programs sponsored by the Office of Economic 
Opportunity (Bricker, 1989; Peterson, 1987; Zigler & Valentine, 1979). This program was 
specifically targeted towards disadvantaged populations. In 1972 and later in 1974, two 
amendments to the Economic Opportunity Act (P.L. 92-424 and P.L. 93-644) impacted the 
availability of educational services to young children with handicaps by mandating Head Stan to 
make ten percent of its total enrollment in each state open to children with special needs (Allen, 
1984). Furthermore, Public Law 94-142 (1975) and its amendment P.L. 99-457 (1986) has 
m?indated educational programs for children wiUi aisabilities from the i^ges three to five years. 
Although many states have provided services for young children since ihe passage of P.L. 94-142 
in 1975, all states will be required to provide services for this age group beginning in the fall of 
1991. 

Activities in IL: Political and Fiscal Atmosphere 

«> 

Historically, Illmois has been one of the leaders in providing EC education for young 
children across a myriad of administrative structures. First, Head Start programs have existed in 
Illinois since 1965. The goals of this program are to provide a comprehensive child development 
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program for preschool aged children including four niajor components: education, health, social 
service, and parent involvement (Illinois State Board of Education, 1990). As previously 
mentioned, the funds for this program were targeted for low-income populations with the main 
purpose to provide children with a "head start" on success prior to entering more formal schooling 
(Bricker, 1989; Peterson, 1987). 

Since 1975, the state of Ulinois has mandated services for young children ages three to five 
years who have special needs (Illinois State Board of Education, 1979; McCollum, 1987). 
Children eligible for early childhood special education (ECSE) services are those who have a 
developmental delay, an established diagnosis (e.g., down syndrome, cerebral palsy), or 
biological risk factors (e.g., medically fragile, HIV positive, prematurity). Federal funds are 
provided to each state through Section 619 or Part B of P.L. 94-142 and most recently P.L. 99- 
457. States distribute these funds to local educational agencies who are responsible for providing 
comprehensive educational programs for three-, four-, and five-year-olds 'Aith disabilities and their 
families. 

As part of the Education Reform Act of 1985, the state of Dlinois through its Department of 
Education, known as the Illinois State Board of Education (ISBE), has developed legislation to 
support public education for an additional group of young children who may be at-risk for later 
academic failure (ISBE, 1990). Eligibility requirements for these prckindergarten programs 
include family income at the poverty level, English not being the primary language in the home, 
parents who are still in their teens or who have not completed high school, and low birth weight or 
prematurity factors which may have resulted in developmental or neurological impa'rment but not 
physical handicap. Over the past five years, the state has continued to increase its financial support 
for these prckindergarten at-risk programs. For example, the FY 1986 Illinois budget allocated 
approximately $12.1 million to serve 5,394 children. In FY 1991. $63 million will serve an 
estimated 25,000 children. The growth of funds and children served show lUinois's strong 
commitment to the education of young children. 

Community child care and preschool programs have expanded as well. Throughout the 
state and nation, communities have provided a proliferation of programs fostering the healthy 
development of young children, especially needed due to the changing nature of families and the 
work force. Not surprisingly, early childhood programs are expanding rapidly, often without 
clear standards or indicators of program quality. Further, many programs have developed in semi- 
isolation as single classes in school districts or conmiunity settings. Few opportunities existed for 
program developers to profit from the experiences of others. This trend of supported growth 



without dc'*umcntation of success or effectiveness, while understandable, does not facilitate the 
development of quality standards for or programs in rarly childhood education (Maude, 1989). A 
strong need existed to identify effective practices in early childhood education tiiroughout the state, 
to describe and investigate those practices, and to publicize them, making descriptions available to 
others developing or wishing to develop programs. ^ 

II. Evaluation Design 

gvaltiatiop P»n?gsg5 

In the Spring of 1990, the intent to conduct a statewide search to identify exemplary 
practices in EC education was announced by ISBE . Three main purposes guided this study: 

1 . By merely conducting the search, the importance of providing quality EC programming 
for all young children would be highlighted. Further public awareness and understanding 
of the nature and scope of EC education throughout the state was intended as an 
outcome of this study. 

2. The search would provide a mechanism to give recognition to exemplary 
practices in partici Jar sites throughout Dlinois. Those individuals providing quality 
programs would be acknowledged publicly on a state and local level. 

3. The search would provide models for other schools or community programs 
developing and expanding services for voung chiWren. As more and more EC programs 
develop, examples of outstanding practices will be useful in tiie dissemination of 
information and adaptation of successful programming. One product developed from this 
search would be a monograph with descriptive information from each of the 

finalists and semi-finalists to serve as a resource for parents, practitioners, and individuals 
preparing to work in or develop new EC programs. 
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Bv^tiation Approach 



The phrase "search for exemplary practices v\ early childhood education'* contains three 
concepts that raise particular issues for evaluators and evaluands and which have important 
implications for the design, implementation, and impact of the evaluation. 

Exemplary. The choice of the term exemplary over alternatives such as best or effective 
was intentional and carefully considered. As defined in Webster*s New World Dictionarv. an 
exemplary practice is one that serves "as a model or example*' (Guralnick, 1987). Because of the 
intent of the evaluation to identify and describe a set of implemented practices that could serve as 
models to others, selection criteria demanded exemplars rather than content-specific criteria or 
standards in early childhood education. 

In the design phase of the evaluation, there was considerable pressure to develop and 
systematically apply a set of standards or indicators of exemplary practice, based in literature and 
research. However, there was no clear consensus among practicing p^fessionals as to what these 
indicators should be. Tremendous variations in what was deemed exemplary stemmed from 
personal and professional philosophy and the type of students served by different EC programs. 
Second, tremendous variation in EC programming existed throughout the state, partly as a result of 
geographic and socioeconomic factors. Illinois is a state with few large populations centers 
(Chicago, Peoria, Rockford, Decatur) and with extensive medical and social services locateu in a 
small area (Chicago, Peoria). Large sections of southern and western Illinois were predominantly 
rural with few population centers over 15,(XX) and limited specialized services. In these areas, 
consolidated school districts serve county-wide or multi-county areas. These differences mitigated 
against the use of a single set of criteria or standards to identify exemplary practice. Finally, given 
the fact that the field of early childhood and especially the field of ECSE is rapidly expanding and 
changing in Illinois and across the nation (McCollum & Maude, in press; McCollum & McCartan. 
1987), any definition of exemplary practice might be somewhat time-limited, needing to be 
reconsidered periodically to account for progress made and to reflect future goals. 

Becj.use this evaluation would need to be sensitive to the real ambiguity of exemplary in 
this context, the investigators decided to use a qualitative, connoisseurship model of evaluation 
(Eisner, 1975). Teams of identified experts in EC, chosen for disciplinary and geographic 
representation, were employed extensively throughout the evaluation. Drawing upon their 
considerab e knowledge in the field of early childhood along with their insight into its special 



context and history in Illinois, these experts provided valuable input in identifying exemplary 
practice. 

Along with the decision to use expert review as the primary means of evaluation, the 
dimensions along which practices were evaluated were chosen to provide evidence that these 
practices would be viable in other areas within the State. 1 he dimensions were: 

(a) program description (e.g., philosophy, program goals, setting, and overall population 
served): To understand the broad context of the program in which the practice was 
operating; 

(b) description of the practice (e.g., services provided and activities conducted, rationale, 
justification for being judged exemplary, cos>s for the practice and the total costs of the EC 
program): To perceive the practice as distinct from the context of the program and to 
estimate the costs involved; 

(c) personnel involvement (e.g., titles, roles, duties, and training of key personnel 
involved in implementing this practice): To describe the nature and extent of 
personnel requirements; 

(d) evidence of effectiveness {e.g., measures and outcome data indicating the effectiveness 
of the nominated practice): To document the impact of the practice; and 

(e) transportability (e.g., how well this practice might work m other areas of the state or the 
comiitions necessary for the practice to be adapted): To ascertain how replicable the 
'practice might be. 

Practice vs. program. The approach to identify exemplary practices rather than 
comprehensive programs was chosen because: (1) it is often easier to replicate a particular practice 
rather than a total model (DeStefano, 1990); and (2) programs may have outstanding practices in 
one or several areas, yet arc still developing as a total model. The search designated seven specific 
practice areas and one undesignated category. A comprehensive Hst of practice areas was produced 
through a review of relevant literature and research and tiirough analysis of practice areas explicitly 
cited in state and federal policy. Using a consensus validation technique in which a panel of 
experts were asked to respona to the relevance and unportance of each area, tiie original list was 
reduced to die seven practice areas used in this evaluation study. A general definition was 
developed for each area. (See Table 1 .) 

Early childhood education. The search was open to all types of early childhood education 
programs, including home- and center-based, regular and special education, and public as well as 



6 

*7 



private. The idea was to "cast a broad net" looking for exemplary practices that would have broad 
applicability to a wide variety of EC programs within commonalities among the types of EC 
programs. 

Evaluation Process 

As stated, a connoisseurship evaluation nKxlcl was chosen as the evaluation approach due 
to the varied nature of EC programming, the irapaa of context on the exemplary nature of practice, 
and the multitude of practice areas under consideration. Teams of EC professionals were actively 
involved across all evaluation phases. In fact, those applications and sites designated as finalists 
underwent a paper or site review by no fewer than eight EC professionals during the evaluation 
process. 

A comprehensive evaluation plan was designed by the authors at the request of Sandra 
Heinzel Crews, the Preschool Training Grant Coordinator at ISBE. This one-year sttidy included 
five major phases: 

a) Phase 1 - peer nominatioi. of reviewers and design and review of instruments; 

b) Phase 2 - application; 

c) Phase 3 - panel review of the applications and selection of semifinalists; 

d) Phase 4 - site review of semifinalists; and 

e) Phase 5 - final panel review and selection of finalists. 

Phase 1 : Peer nomination o f reviewers and instrument design. In August of 1990, ISBE 
issued a memo to all EC professionals in the State, requesting nominations of individuals viewed 
as "experts" in the field of early education. Over 200 responses identified individuals in EC 
education and ECSE: administrators, personnel trainers at instittites of higher education, parents, 
service delivery practitioners, and supervisors. These nominees were associated with public, 
private, and community agencies throughout the state. This pool of professionals and parents was 
tapped for participation throughout all five phases of the evaluation study. 

Several instruments were developed, reviewed, and piloted: the initial application, panel 
review forms, site review training packet, and the final review instrument. A group of nominated 
experts was asked to critique all instruments and to comment upon the proposed design. 

Phase 2: Application . The application packet (Appendix A) was designed during the 
summer of 1990. A draft was reviewed by nominated experts and revised based on then- input. 
The application included seven main sections. First, an overall description of the search was 



provided including purposes, a general definition of exemplary practice, specification and 
definition of each of the seven practice areas, evaluation questions, and responsibilities of the 
finalists. The second section requested demographic information from each applicant such as 
contact names and addresses, tai:get population, funding source(s), service location, and number of 
children served. At this point, the applicant was requested to identify the practice area nominated, 
it being clearly indicated that individuals or service delivery programs could submit applications in 
more than one practice area; however, only one application per practice area per applicant would be 
considered. 

The remaining five sections of tiie application asked the applicant to provide information for 
each of the five evaluative dimensions: (a) program description; (b) description of the practice; 
(c) personnel involvement, (d) evidence of effectiveness; and (e) transportability. Finally, each 
applicant was asked to provide an authorized signature on the completed application by an 
administrator or school superintendent, thereby indicating support by the administration as well as 
informing them of the evaluation process. 

In September, 1990. nearly 5.000 applications were sent to public and private programs 
throughout die state to solicit nominations of exemplary practices in the eight categories. 
Addressees included public and private service providers, administrators, and practitioners 
providing direct services. (See Table 2.) Potential applicants were notified that they might submit 
an application, due to ISBE by die end of October, 1990. Over 1 15 programs were represented 
among the applicants on an individual or co-author basis. The largest source of applications 
submitted were from state-funded prekinderganen at-risk Qi = 49) and ECSE programs (H = 36). 
(See Table 3.) The total number of indi\idua! applications received was 93. witfi the greatest 
number of nominations submitted in die categories of program design (M = 27) and family 
involvement (H ~ 21) with die smallest number in administration (N=6), other (N=3), and cultural 
promotion Qi=2). (See Table 4.) The applications ranged in lengtii from five to thirty pages. 
Applicants provided a wide variety of information such as videotapes, evaluation reports, copies 
of curricula, and parent testimony to describe their practice and to provide illustration of its 
exemplary nature. 

Phase 3: Panel review and se lection of the semifmalists. In November of 1990, a two-day 
review session was held in Springfield, Illinois. Fifteen EC and ECSE professionals in five three- 
r«ember panels reviewed the 93 applications. Each panel read between 17 to 21 proposals in one 
to three practice areas. Proposals were rated by individual panel membws who assigned points 
from die following maximum distribution: program description (5 points); description of practice 
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(40 points); personnel involvement (15 points); evidence of effectiveness (20 points); and 
transportability (20 points) for a total possible score of lOO points. (A copy of the evaluation form 
used for the ratings is included in Appendix B.) 

Following the individuals* reviews, each three-person panel met to discuss their individual 
ratings and to reach a consensus score. Each panel score was then standardized onto a z-score 
scale to neutralize different scoring tendencies by panels (i.e., so no applicant would be penalized 
because its reviewing panel tended to score lower than other panels). Each panel was also asked to 
make a global recommendation of whether the practice was exemplary or not These 
recommendations were used to set a cut-off score for identification of exemplary practices. By 
applying a cut-off score of z = 1.0, nineteen applicants were identified as semi-finalists. The 
largest number of semi-finalists were chosen in the area of program design (d = 6), followed by 
family involvement (Ii= 5), service delivery (H = 2), integrated setting (M, = 1), and staffing 
patterns (N = 3). Two practices were selected in the "other" category; no practices were selected 
under administration or cultural promotion. (See Table 4.) All applicants were contacted and 
thanked for their participation in the project and the semi-finalists moved into phase four of the 
evaluation study. 

Phase 4: Site review process . A unique aspect of this evaluation study was the inclusion 
of an on-site review phase. The purpose of the on-site review was not only to confirm and clarify 
the information presented in the initial application, but to provide additional supporting evidence 
regarding the exemplary nature of the practice. Acknowledging Uiat each practice and program 
might have unique aspects and ircumstances, no rigid protocol for site review was presented. 
Instead, teams of two reviewers were called upon to interview, observe, review documents, and to 
present findings in a narrative site visit report The specific content and nature of data collection 
activities were specific to each site, determined by the siie visitors in collaboration with the 
program staff. (See Appendix C for tiie site review protocol, agenda, and report form.) 

A three-step process characterized die site visits. First, in January 1991, nineteen 
nominated experts, parents and professionals, were contacted to participate in a one-day site review 
training. Training for tiiese site reviewers included an orientation to the evaluation, an overview of 
basic qualitative evaluation methods, a review of the applications, and instructions on bow to set 
up a site visit and how to write die final report Each site was visited for one da> by a panel of 
two team members. The panel members were selected tc represent different professional 
backgrounds and experience. Panels were assigned to review programs in geographic regions 
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outside their own. No panel member reviewed a program with which he or she had any personal 
or professional association. 

Site reviewers were responsible for contacting their sites and negotiating an agenda that 
would provide opportunities for collecting data relevant to the guiding questions. Typical activities 
might include interviews with vhc program administrator, teachers, selected staff members, and 
consumers; observation of the practice; discussion with parents; and review of documents. 

Within five days of the completion of the site review, the panel was asked to submit a 
written site report Four questions guided the site visit and the final report: 

1 . To what extent did the site visit confirm information presented in the initial 
application? What clarifications were made? 

2 . What additional information can now be provided to support or refute the 
exemplary nature of this practice? 

3. What can be cited as evidence that this practice is effective? In your best 
judgment, how will this practice work in other areas? What factors are 
necessary to insure its success? What are the barriers to implementation of 
this practice? 

4 . What are your overall impressions of the practice? 

Thes; reports were submitted in late February 1991 and a follow-up debriefing with each reviewer 
was conducted by phone prior to the final phase. 

Phase 5 - Final panel review and selection. Anotiier panel of reviewers was convened in 
early March 1991 to review the application materials and the site visit reports and to select the 
finalists. 

Hit Evaluation Results 

Nine programs were selected as finalists in the "Search for Exemplary Practices in Early 
Childhood Education." (See Table 4.) These finalists represented four practice areas: staffing 
patterns, service delivery, family involvement, and program design. No finalists were selected in 
the areas of integrated settings, administration, cultural promotion, and the other category. 

Finalists selected for recognition as exemplary practices were awarded small grants to help 
them share information as models witii otiters interested in adopting tiiese practices. Upon 
selection, their first activity was to provide a brief description of Uie practice for a resource 




directoiy tentatively entitled, "Exemplary Practices in Early Childhood Education." Finalists were 
also asked to disseminate materials about the exemplary practice; to conduct training sessions on 
their exemplary practices at a statewide conference; and» depending upon funds available, to 
provide technical assistance to interested parties. 

IV. Conclusions 

Comments on the Evaluation Approach 

Tne decision to use professional review to identify exemplary practice rather than a priori 
standards or criteria suited the absence of commonly shared philosophies, resources, experiences, 
and standards. The high level of professional and parental input and interaction surrounding this 
evaluation may have moved the state closer to an understanding of the commonalities that exist in 
programs for young children across ability groupings, service providers, and geographic areas. 
While the use of a priori standards in this evaluation might have been easier to implement and less 
labor-intensive, the absence of conmion agreement about standards might have alienated some 
potential applicants. The use of professional judgment seemed to give programs ample opportuniiv 
to convey their philosophies and evidence of effectiveness through the use of self- nomination and 
on-site visits. 

Because the evaluation provided the opportunity for a large group of experts from differtni 
professional and personal experiences to become familiar with EC services and new pracrices and 
programs throughout the state, a recommended next step might be to reconvene expen participants 
for a one- or two-day work group, for the purpose of developing a set of general ♦ idicators or 
standanls of best practice for EC. Secondarily, a large group of influential experts within the state 
has begun to develop a network that cuts across service providers, regions of the state, and types 
of students served. 

There has been another unanticipated outcome attributable to the evaluation approach used 
here. The use of content experts in the planning, design, and conduct of the evaluation helped to 
build credibility and ownership in what could have been a highly competitive and political 
situation. The evaluation client, reports few complaints to her office regarding the outcomes of the 
evaluation project 
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The inclusion of several layers of expert review i n all instruments and design cations, the 
additional time needed to train site vif^itors in qualitative methods and reporting, the massive 
amount of paper documentation that accrued foi each pp>ject by the end of the evaluation process, 
and the hours of phone :;alls required to coordinate busy peoples' schedules and to arrange for site 
visits and training and review sessions all mitigate against the use of this approach when time, 
money, and personnel resources are not available. 

Comments on the Use of Case Study Methodology 

Initially, there was some apprehension among the evaluators and the evaluation client 
concerning novice evaluators* use of case smdy or qualitative methodology. While some experts 
had had experience with site review, most of that experience involved compliance monitoring and 
formalized reporting using checklists. It was a pleasant surprise to discover that with training, 
experts could produce high quality case smdy reports. In the debriefing, virtually all site reviewers 
said that the experience was worthwhile and provided them with additional infonnation that 
supported or refuted the exemplary nature of the practices. Some said that they wanted to use case 
studies jn their own future work. The case study information was critical in the selection of 
finalists. Case study infonnation will also be used in the resource directory that will be made 
available to persons interested in implementing practices in otho" areas. 

All experts selected for site review (including parents) knew of the interpersonal, travel, 
writing, and time demands before agreeing to participate. Several people declined participation 
because of tiiese demands. If it is critical to have particular stakeholder groups, such as parents, 
participate in the evaluation and if it is likely that these demands preclude their participation, 
modifications may be possible. For example, instead of requiring the submission of a written 
report, site reviewers may choose to make an audiotape of their report If parents or others feel 
uncomfortable interviewing professionals, they may be given input into Uie development of an 
interview protocol and U'.en asked to respond to transcripts or tapes of the interviews after tiiey 
have been completed. 
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Comments on the Importance of the Evaluation to the State 

It is too early to judge the impact of the evaluation within the state. To date, evaluation 
results have not been disseminated widely. Only ISBE personnel, including the evaluation client 
and the finalists are aware of the final outcome of the year-long activity. Even at this early siage, 
however, there have been some interpretadons given to the findings and some decisions made. 

It is generally agreed that the types of piacticcs selected can be used as a profile of the 
development of early childhood in Illinois. Most local activity seems to be concentrated in the 
practice areas of sendee delivery, staffing, program design and family involvement, the areas 
receiving the most applicants, senu-fmalists, and finalists. This pattern is not surprising as (1) 
family involvement has been heavily mandated at both federal and state levels- and (2) many 
programs are quite new, hence program design may account for a large amount of exemplary. 
Interestingly, two semi-finalists were selected from the other category, suggesting growth in areas 
not specified by research or policy. Certainly these practices warrant further study. 

The evaluation also revealed that good things were happening throughout the state, even in 
locales challenged by large catchment areas and limited availability of professionals and services. 
This finding was welcomed with the hope that replication of exemplary practices will be possible in 
all parts of the state. 

Without a doubt, the evaluation dimension that most of the applicants had difficult 
responding to was the sections asking for evidence of effectiveness. It became clear that local 
programs did not have access to data concerning tht effectiveness of their programs. Possible 
reasons for this were suggested. It may be that, facwd with the need to provide services and to be 
in compliance witii state law, newly developing programs "implement now and evaluate later" 
instead of incorporating evaluation into program planning and implementation. One problem with 
the "implement now and evaluate later** strategy is Uiat practices do not have evidence of 
effectiveness readily available during program development and implementation. It is also possible 
that local program personnel lack the skills, resources, and encouragement to evaluate their 
practices. One possible action tiiat the state could take is to provide training and small grant 
funding to support evaluation in developing programs. 

It is too soon to judge how well Uie practices identified as finalists will serve as models but 
their impact will be monitored by ISBE. Dissemination began at the fu^t statewide conference on 
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early childhood in mid-April of 1991 when finalists were announced and asked to make 
presentations about their selected practices. 

Implications o f the Evaluation for Other States and Content Areas 

Overall the evaluation approach used in this study seems to be good way to assess 
exemplary practices in a developing field. Expert involvement and consensus building increases 
ownership and commitment to the evaluation as well as identifying a substantial group of experts 
who may be used to describe practice and develop standards in subsequent activities. The 
inclusion of site visits and case study methodology adds time and money but produces the benefit 
of increased validity of the study. 
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Table 1 

Practice areas and definitions. 



1 . Administration: 

2. Cultural Promotion: 

3. Family-centered: 



4. Integrated Setdngs/ 
Alternative Service 
Delivery Settings: 



5. Program Design: 



6. Service Delivery: 



7 . Staffing Patterns and 
Staff Development: 



8. Other 



Managerial and leadership that supports and guides the early 
childhood program. 

Emphasis and sensitivity to a child's cultural and linguistic 
variations. 

Active* individualized participation by parents and family 
members in their child's education. 



Settings that intcpate children with disabilities and 
norm^y developing peers in typical and/or unique settings. 

A clearly articulated philosophy, well defined theoretical 
orientation about development and learning in young 
children, and goals and objectives tnat are designed for the 
program, children, and families. Strong linkages exist 
between philosophy, tiieoretical orientation, and goals 
and objectives in designing a comprehensive early 
childhood program. 

Selected strategies and processes are used to support the 
specific service delivery approach(es) utilized 
(e.g., assessment, intervention, evaluation, transition, 
family support, community education, among others). 



Qualified professionals providing services in a collaborative, 
coordinated delivery system. 
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Table 2 

Distribution sources for application materials. 



Source 

Head Start 82 
Prekii dergarten At Risk 250 

Community day care/preschool 1 ,422 

R*TAS mailing list 2,221 

(inservice training for ECSE) j 
District superintendents 872 | 

Directors of special education 1 1 2 



TOTAL 4.959 
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Tables 

Breakdown of applicarions received 



Source ^ 

State funded Pre-K at risk 49 
Early Childhcxxi Special Education 36 
Head Start 9 
Other private/community 20 

Total 115* 



♦The total number on this table exceeds the number of applications received by practice area 
(M=93) due to a number of applications submitted jointly by two or more programs. 



Table 4 

Breakdown bv practice area 



Practice Area 


Number of 
AppUcants 


Number of 
Semifinalists 


Finalists 


Program Design 


27 


6 


2 


Family Invoivement 


21 


5 


3 


Service Delivery 


18 


2 


2 


Integrated Settings 


8 


1 


0 


Staffing Patterns 


8 


3 


2 


Administration 


6 


0 


0 


Other 


3 


2 


0 


Cultural Promotion 


2 


0 


0 




93 


19 


9 
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